Automatic Recognition of Mexican Sign Language Using a Depth Camera and Recurrent Neural Networks

نویسندگان

چکیده

Automatic sign language recognition is a challenging task in machine learning and computer vision. Most works have focused on recognizing using hand gestures only. However, body motion facial play an essential role interaction. Taking this into account, we introduce automatic system based multiple gestures, including hands, body, face. We used depth camera (OAK-D) to obtain the 3D coordinates of motions recurrent neural networks for classification. compare model architectures such as Long Short-Term Memories (LSTM) Gated Recurrent Units (GRU) develop noise-robust approach. For work, collected dataset 3000 samples from 30 different signs Mexican Sign Language (MSL) containing features face, hands spatial coordinates. After extensive evaluation ablation studies, our best obtained accuracy 97% clean test data 90% highly noisy data.

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Recognition of sign language gestures using neural networks

This paper describes the structure and performance of the SLARTI sign language recognition system developed at the University of Tasmania. SLARTI uses a modular architecture consisting of multiple feature-recognition neural networks and a nearest-neighbour classifier to recognise Australian sign language (Auslan) hand gestures.

متن کامل

Exploiting Recurrent Neural Networks and Leap Motion Controller for Sign Language and Semaphoric Gesture Recognition

In human interactions, hands are a powerful way of expressing information that, in some cases, can be used as a valid substitute for voice, as it happens in Sign Language. Hand gesture recognition has always been an interesting topic in the areas of computer vision and multimedia. These gestures can be represented as sets of feature vectors that change over time. Recurrent Neural Networks (RNNs...

متن کامل

Sign language perception research for improving automatic sign language recognition

Current automatic sign language recognition (ASLR) seldom uses perceptual knowledge about the recognition of sign language. Using such knowledge can improve ASLR because it can give an indication which elements or phases of a sign are important for its meaning. Also, the current generation of data-driven ASLR methods has shortcomings which may not be solvable without the use of knowledge on hum...

متن کامل

Robust Sign Language Recognition System Using ToF Depth Cameras

Sign language recognition is a difficult task, yet required for many applications in real-time speed. Using RGB cameras for recognition of sign languages is not very successful in practical situations and accurate 3D imaging requires expensive and complex instruments. With introduction of Time-of-Flight (ToF) depth cameras in recent years, it has become easier to scan the environment for accura...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Applied sciences

سال: 2022

ISSN: ['2076-3417']

DOI: https://doi.org/10.3390/app12115523